Computer networks such as the Internet allow computer systems to exchange content (e.g., data or information such as documents, web pages, files, database information, streams of audio and/or video data, or the like) in a variety of ways. For example, computer systems (e.g., client computer systems, server computer systems) exchange content over the computer network by using a suite of protocols collectively referred to as the World Wide Web. In a typical operation of the World Wide Web, a client computer system, configured with a web browser, requests content served by a web server computer system over the Internet. The web server transmits the requested content over the Internet to the requesting client computer system.
Client computer systems that require the ability to access content using the aforementioned techniques may be connected to a computer network at diverse geographic locations. In such situations, a producer, provider or other originator of the content may use a group of technologies, collectively called a content distribution network (CDN), to disseminate or distribute the content.
A CDN enables web content from an origin server to be distributed to other servers at various locations in the network for storage and access. A CDN typically includes a content distribution manager (CDM) to manage the CDN, at least one content router (e.g., operating as a load balancer) and one or more content engines that operate as servers (e.g., web servers) to serve content requested by content requests sent from client computer systems. The content router receives client requests for content, and using various content routing techniques, decides which content engine is to service those client requests. The content router then redirects the client requests to an appropriate content engine. The content engines are also called content servers, or alternatively, caching servers. A content engine that receives redirected requests from a client computer system typically operates to serve the requested content back to those client computer systems, though the content engine may perform further redirection techniques as well in some instances.
When a user requests a Web page, for example, that is part of the CDN, the CDN typically redirects the request from the origin server to a content engine that is closest to the user. This redirection may be based on a number of factors such as the geographic proximity of the requesting client to an available content engine containing the requested content, or on the specific content requested, or on load balancing considerations between multiple available content engines. The content engine delivers the cached content to the user. The CDN also communicates with the origin server to deliver any content that has not been previously cached.
As an example of the operation of a CDN, a user controlling a web browser operating on a client computer system may select a Uniform Resource Locator (URL) that references a specific portion of content served by the CDN. A domain name specified within the URL might generally reference the CDN itself, such as www.CDN.com and protocols such as the Domain Naming System (DNS) can initially direct that client content request to a content router associated with that CDN. Through various redirection techniques (e.g., Domain Naming System redirection and/or Hypertext Transport Protocol redirection), the content router that receives the client request for content can redirect this client request to an address of a specific content engine within the CDN based on the aforementioned routing criteria (e.g., the geographic location of the client providing the client content, load balancing considerations made between the various content engines, and so forth). A content engine that receives the redirected content request can then process this request to serve the requested content to the client computer system.
As explained above, a conventional CDN accelerates access to content on behalf of client computer systems that request the content by distributing the content among content engines distributed throughout a computer network. Rather than all client computer systems requesting the same content from a single origin server, individual client computer systems are able to obtain the requested content from a content engine that is “close” to the client and that locally stores the required content. This improves a client's performance with respect to access to the content for a number of reasons. The client computer system may be much closer in network terms (e.g., number of network hops) to the content engine as compared to the origin server. In addition, the data communications link from the client computer system to the content engine may be a high-speed connection while one or more data communications links (e.g., a satellite link) along the network path from the client computer system to the origin server may be relatively slow data communications links. As a result, the client computer system can access the content faster (i.e., can get better response) from the more local content engine than from the more distant or remote origin server. In addition, by having many content engines distributed throughout a computer network, the load, demand and bandwidth requirements of each content engine is generally reduced as each content engine needs to serve content to only certain client computer systems.
Various conventional techniques and mechanisms also exist for distributing the content from the origin server to the selection of content engines distributed through the computer network in order to accelerate access to the content by client computer systems. One technique for distributing content is the content channel. A content provider defines one or more “channels” to make content available through the CDN. Each channel contains a set of files that is typically expected to be accessed by the same set of users. For example, a channel may contain training videos intended to be used by sales people, or advertisements directed at an organization's customers. A subset of the content servers in the CDN is assigned to each channel. The content servers in the assigned subset are typically servers located conveniently with respect to the intended set of users of the channel.
Files in a channel are pre-positioned at the content servers assigned to that channel. Pre-positioning enables greater certainty of availability of the content to accessing users. The content servers assigned to the channel may be behind a slow link relative to the origin server, and the files themselves may be very large. Therefore, moving the content from the origin server to a content engine assigned to the channel can be time-consuming. Pre-positioning the content generally reduces delay when users attempt to access files. Pre-positioning content within content engines is especially important when the content is very large or in situations where there is a low speed connection between the origin server and the content engine that will cache or otherwise store the content for access by requesting client computer systems. By pre-positioning the content, client computer systems encounter minimal delays when requesting the content from the content engines.
Routing within the CDN may be further accelerated by creating an overlay network. An overlay network includes a collection of nodes placed at strategic locations in an existing network fabric, also referred to as a substrate network. The overlay network further includes specified connections between the nodes in the collection of nodes. Together the collection of nodes and the specified connections implement a network abstraction on top of the substrate network. Routing is accelerated by selecting optimal nodes and connections. In a CDN, the overlay network includes the content engines and connections between the content engines. The connections are selected in order to optimize data transfer between the content engines.